Papers with tiny task model
UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have limited applicability for inference due to their large parameter size and limited generalization ability. |
| Approach: | They propose a method that generates a dataset regardless of the target domain . this allows for generalization of the tiny task model to any domain that shares the label space . |
| Outcome: | The proposed method achieves generalizability across domains while using a parameter set that is orders of magnitude smaller than PLMs. |
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios. |
| Approach: | They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset. |
| Outcome: | The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation. |